Skip to content

gh-148829: Implement PEP 661#148831

Open
JelleZijlstra wants to merge 16 commits intopython:mainfrom
JelleZijlstra:pep661
Open

gh-148829: Implement PEP 661#148831
JelleZijlstra wants to merge 16 commits intopython:mainfrom
JelleZijlstra:pep661

Conversation

@JelleZijlstra
Copy link
Copy Markdown
Member

@JelleZijlstra JelleZijlstra commented Apr 21, 2026

Comment thread Objects/sentinelobject.c Outdated
Comment thread Lib/typing.py
Comment thread Objects/sentinelobject.c Outdated
Comment thread Objects/sentinelobject.c Outdated
Comment thread Objects/sentinelobject.c Outdated
Comment thread Objects/sentinelobject.c Outdated
Comment thread Objects/sentinelobject.c
Comment thread Objects/sentinelobject.c
Comment thread Objects/sentinelobject.c Outdated
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
@sunmy2019 sunmy2019 self-requested a review April 23, 2026 19:42
Comment thread Doc/c-api/sentinel.rst Outdated
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
@JelleZijlstra JelleZijlstra added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 24, 2026
@bedevere-bot
Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @JelleZijlstra for commit 53a84b6 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F148831%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 24, 2026
Copy link
Copy Markdown
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The C code and the corresponding documentation look good to me. I didn't look very closely at the tests.

Comment thread Objects/sentinelobject.c
assert(PyFunction_Check(func));
PyObject *r = PyFunction_GetModule((PyObject *)func);
if (!r) {
Py_RETURN_NONE;
Copy link
Copy Markdown
Member

@sunmy2019 sunmy2019 Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need a PyErr_Clear() here? (since if error occurs, usually we will return NULL)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ZeroIntensity asked me to remove it (#148831 (comment)) because there is no scenario where this function can return NULL without an exception set.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, PyFunction_GetModule can only ever raise if the input argument is not a function, which we know will never be true here. We could add an assert(!PyErr_Occurred()) if you'd like.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for assert(!PyErr_Occurred())

Copy link
Copy Markdown
Member

@sunmy2019 sunmy2019 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Copy Markdown
Member

@sunmy2019 sunmy2019 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarify the pickle document. So that people will see X = sentinel("Y") is not picklable.

Comment thread Doc/c-api/sentinel.rst
Comment on lines +28 to +29
*module_name* should be the name of an importable module if the sentinel
should support pickling.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
*module_name* should be the name of an importable module if the sentinel
should support pickling.
For pickling to work, *module_name* must be the name of an importable
module, and the sentinel must be accessible from that module under a
path matching *name*. Pickle treats *name* as a global variable name
in *module_name* (see :meth:`object.__reduce__`).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding this would help people use pickle properly. People need to assign this sentinel instance to that module before pickle.dump.

Comment thread Doc/library/functions.rst
Comment on lines +1844 to +1846
Sentinels importable from their defining module by name preserve their
identity when pickled and unpickled. Sentinels that are not importable by
module and name are not picklable.
Copy link
Copy Markdown
Member

@sunmy2019 sunmy2019 Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Sentinels importable from their defining module by name preserve their
identity when pickled and unpickled. Sentinels that are not importable by
module and name are not picklable.
Sentinels are pickled using the standard mechanism for named global
objects. :meth:`sentinel.__reduce__` returns :attr:`~sentinel.__name__`,
which pickle interprets as the name of a global variable in
:attr:`~sentinel.__module__`. Sentinels that are not accessible from their
module under that name are not picklable.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I agree we should expand more but feel it might be a bit much to say exactly how __reduce__ is implemented. I'll get back to this tonight and push some changes.

@JelleZijlstra
Copy link
Copy Markdown
Member Author

JelleZijlstra commented Apr 24, 2026

Buildbot run: A few failures look unrelated

In file included from ./Include/internal/pycore_backoff.h:12,
                 from ./Include/internal/pycore_code.h:11,
                 from ./Include/internal/pycore_call.h:11,
                 from ./Modules/_ctypes/_ctypes.c:109:
./Modules/_ctypes/_ctypes.c: In function ‘set_stginfo_ffi_type_pointer’:
./Modules/_ctypes/_ctypes.c:2234:40: error: ‘FFI_TYPE_COMPLEX’ undeclared (first use in this function); did you mean ‘FFI_TYPE_DOUBLE’?
         assert(fmt->pffi_type->type == FFI_TYPE_COMPLEX);
                                        ^~~~~~~~~~~~~~~~
./Modules/_ctypes/_ctypes.c:2234:40: note: each undeclared identifier is reported only once for each function it appears in

But I also see

Traceback (most recent call last):
  File "/home/ubuntu/buildarea/pull_request.stan-aarch64-ubuntu.oddballs/build/Lib/test/pickletester.py", line 3254, in test_builtin_types
    s = self.dumps(t, proto)
  File "/home/ubuntu/buildarea/pull_request.stan-aarch64-ubuntu.oddballs/build/Lib/test/test_xpickle.py", line 190, in dumps
    return self.send_to_worker(python, data)
           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/home/ubuntu/buildarea/pull_request.stan-aarch64-ubuntu.oddballs/build/Lib/test/test_xpickle.py", line 172, in send_to_worker
    raise exception
AttributeError: Can't get attribute 'sentinel' on <module 'builtins' (built-in)>

https://buildbot.python.org/#/builders/1865/builds/54

Which is a bit mysterious, is this looking at the builtins of another build?

@sunmy2019
Copy link
Copy Markdown
Member

Which is a bit mysterious, is this looking at the builtins of another build?

Yes. We need to patch this test. It tests pickle compatiblity acrosses python versions.

./python -m test test_xpickle -m test_builtin_types -u xpickle

@sunmy2019
Copy link
Copy Markdown
Member

diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py
index 6366f12257..c2018c9785 100644
--- a/Lib/test/pickletester.py
+++ b/Lib/test/pickletester.py
@@ -3244,6 +3244,7 @@ def test_builtin_types(self):
             'BuiltinImporter': (3, 3),
             'str': (3, 4),  # not interoperable with Python < 3.4
             'frozendict': (3, 15),
+            'sentinel': (3, 15),
         }
         for t in builtins.__dict__.values():
             if isinstance(t, type) and not issubclass(t, BaseException):

@sunmy2019
Copy link
Copy Markdown
Member

Buildbot run: A few failures look unrelated

I opened #148967 to track it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants